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Abstract: Users of augmentative and 

alternative communication (AAC) aids could 
benefit from novel methods for accelerating 
access to contextually relevant vocabulary. This 
paper describes our initial efforts toward 
improving access to situational vocabulary 
through the use of geographic context to 
predict vocabulary. A corpus of spoken data 
produced by one able-bodied male speaker 
across various locations was collected and used 
to mine location-specific vocabulary clusters. 
We describe the data mining algorithm and 
illustrate how context-driven vocabulary 
organization and prediction can be integrated 
into an iconic communication system. 
Correlations between user and algorithm- 
generated vocabulary patterns highlight 
discrepancies between actual versus perceived 
frequency statistics. Thus, device customization 
may be achieved through a hybrid human- 
algorithmic approach to vocabulary selection. 
Endowing the communication aid with 
knowledge about the user’s location-specific 
vocabulary patterns may help to balance the 
burden of communication between the system 
and the user to yield substantial gains in 
accessing situationally appropriate vocabulary 
which may in turn accelerate communication 
rate. 

Keywords: Augmentative and alternative 

communication, Fringe vocabulary, Vocabulary 
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Communication is a slow and effortful process 
for nearly two million Americans with severe 
speech impairments who must rely on AAC 


devices. Communication efficiency is an 
especially important issue for AAC users who 
are fluent in the language and thus require 
access to large vocabularies. Current AAC 
devices employ a number of techniques to 
enhance access to frequently used words 
including message encoding, semantic 
compaction, and dynamic layouts. All of these 
methods, however, have practical limits in 
terms of the number of words that can be 
retrieved and the cognitive burden imposed on 
the user. While a significant portion of an AAC 
user’s message consists of core vocabulary 
items which are common across contexts 
(Beukelman, Yorkston, Pobleto, & Naranjo, 
1984), situational vocabulary (also referred to as 
fringe or extended vocabulary) may change 
substantially as the user encounters different 
topics and settings throughout a day. Although 
access to situational vocabulary is essential for 
engaging in timely and relevant conversations, 
it imposes additional challenges when designing 
efficient visualization and navigation schemes. 
Leveraging knowledge about the user’s 
geographic context to guide situational 
vocabulary prediction has the potential to 
improve communication efficiency and thereby 
contribute to optimizing educational, social, 
and vocational opportunities. This paper 
reports on an initial exploration aimed at 
harnessing location-specific vocabulary usage 
patterns to improve access to less frequently 
used situational vocabulary. 
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Background 

Communication using AAC devices is 
significantly slower and effortful compared to 
typing or speaking (Beukelman & Mirenda, 
1998; Goldman-Eisler, 1968; Szeto, Allen, & 
Littrell, 1993). This mismatch in 
communication efficiency between AAC users 
and their able-bodied peers (a) impacts upon 
the number and type of communication 
opportunities available (Demasco, 1994; 

Higginbotham & Wilkins, 1999; Light, Collier, 
& Parnes, 1985; Muller & Soto, 2002; Simpson, 
Beukelman, & Sharpe, 2000); (b) contributes to 
perceptions of cognitive incompetence (Beck, 
Kingsbury, Neff, & Dennis, 2000; 

Goodenough-Trepagnier, Galdieri, Rosen, & 
Baker, 1984; Light 1985, 1988; Hoag, 

Bedrosian, Johnson, & Molineux, 1994); and (c) 
limits social and vocational options for AAC 
users (Beck & Dennis, 1996; Beck, Fritz, 
Keller, & Dennis, 2000). 

In an effort to speed and ease communication 
most current AAC devices employ some 
method of enhancing access to frequently used 
words, including, dynamic layering or thematic 
organization of vocabulary (Demasco, 1994; 
Wilson & Fox, 1993), message encoding 
schemes (Light & Lindsay, 1992), and Semantic 
Compaction™ (Baker, 1982, 1986). A common 
metric used for assessing enhanced access to 
vocabulary items is keystroke savings (see 
Higginbotham, 1992 for a comparison of 
keystroke savings across several AAC devices). 
In other words, measuring the number of 
physical entries required to construct a 
message, with and without a given access 
method. Semantic Compaction™, an encoding 
scheme in which each icon has multiple 
meanings, has been shown to improve 
efficiency by reducing the item set on a visual 
display (Baker, 1987). Similarly, letter and 
number encoding schemes in which particular 
sequences of letters and/or numbers activate 
words or messages reduce keystrokes (Baker, 
1982, 1986; Gardner-Bonneau and Schwartz, 


1989; Vanderheiden & Lloyd, 1986). Although 
encoding schemes may be easier to process and 
recall, they impose limitations on the size of 
vocabulary set for reasons of practicality and 
cognitive load and thus may not be appropriate 
for fluent users who require access to a large 
and diverse vocabulary set. 

In order to accommodate larger vocabulary sets 
that cannot be visualized at once, some devices 
employ dynamic layouts in which single 
meaning icons are organized into semantically 
related clusters. To provide the user with a 
sufficiently rich lexicon, it is essential to have 
multiple vocabulary pages. There are trade-offs, 
however, between vocabulary size, the 
cognitive load of vocabulary management, and 
the physical load of vocabulary selection. 

While the above methods can improve access 
to core vocabulary items, the overall gain is not 
sufficient to place AAC users on par with their 
able-bodied peers. AAC users also use words 
that are not in this core. Beukelman, Yorkston, 
Pobleto, and Naranjo, (1984) reported a core 
vocabulary of 500 words based on 
conversational samples across five non- 
speaking adults. Across the five participants, 
only 33% of any given user’s communicative 
utterances could be generated using only those 
500 core vocabulary items. One potentially 
effective strategy for improving vocabulary 
access for fluent AAC users may be to focus on 
situationally-dependant words that a user may 
need in everyday communication. Endowing 
the communication aid with knowledge about 
the user’s vocabulary patterns in his/her daily 
routines may balance the burden of 
communication between the system and the 
user. 

Word prediction is one method of enhancing 
access situational vocabulary (c.f. Aim, Arnott, 
& Newell, 1992; Carlberger & Hunnicutt 1998; 
Copesake, 1997; Foulds, Soede, & van Balkom, 
1987; Hunnicut, 1986; Newell, Arnott, Beattie, 
& Brophy, 1992; Swiffin, Arnott, & Pickering, 
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1987). Traditional word prediction approaches 
use statistical information such as frequency 
and recency of use to order and present a list of 
the most likely lexical candidates. Although 
there is almost no learning curve in using word 
prediction systems compared to encoding 
schemes, it is not clear whether the keystrokes 
savings of word prediction translates into 
improved communication rate or access to the 
appropriate vocabulary (Higginbotham, 1992; 
Koester & Levine, 1994, 1997; Lesher, 

Moulton, & Higginbotham, 1998; Venkatagiri, 
1994). The costs incurred by switching between 
selecting icons or letter and scanning or reading 
lists of possible items can sometimes outweigh 
the benefit of reduced keystrokes (Koester & 
Levine, 1994; Treviranus & Norris, 1987; 
Venkatagiri). 

Especially when the vocabulary size is large, 
word prediction systems based on a database of 
word frequencies and inter-word correlations, 
have a hard time predicting the word that the 
user actually intended. To address this issue, 
Lesher and Rinkus (2001) proposed that 
domain-specific vocabularies may help to 
constrain the predicted item list. They 
measured the average keystroke savings of an 
n-gram prediction model with and without the 
use of domain-specific vocabularies derived 
from the Switchboard corpus of telephone 
transcripts. They demonstrated that endowing 
the system with domain-specific knowledge can 
afford keystroke savings between 53-58% 
relative to models that do not incorporate such 
information. Constraining the word prediction 
to offer situationaly-appropriate vocabulary 
items may decrease the cognitive dissonance 
associated with searching lists of unrelated and 
unnecessary words (Renaud, 2002; Venkatagiri, 
1994). Since situational vocabulary can 
potentially be enormous, it requires new ways 
to thinking about how to make it available to 
AAC users such that their interactions can be 
timely and relevant to the topic and setting of 
conversation. 


The idea of using domain-specific vocabulary 
has made it into some AAC products (e.g., 
SOLO™ by Don-Johnston, Inc.); however, 
context is predetermined and vocabulary is 
predefined by the user or his/her clinician. 
Balandin and Iacono (1998) showed that even 
trained professionals have difficulty predicting 
vocabulary usage. They asked ten professionals 
(five speech pathologists, three rehabilitation 
counselors and two teachers) to predict topics 
and vocabulary of meal break conversations at 
work. While the professionals accurately 
predicted some conversational topics, 
approximately 33% of the key words predicted 
by the participants did not occur in the 
conversational sample at all. Thus predicted 
and actual usage patterns may not always be 
aligned. Moreover, within a given setting, 
conversational topics can be diverse. Balandin 
and Iacono (1998) collected meal break 
conversational samples from 34 non-disabled 
participants across four work sites and found 
that the participants referenced 73 different 
topics some of which were associated with the 
day of the week. Upon further analysis of the 
conversational sample, Balandin, Iacono, and 
Crews (1999) reported that a small stable core 
vocabulary of approximately 350 words 
accounted for 78% of the conversational 
sample. Fluent AAC communicators may have 
equally diverse conversations and thus the 
challenge is to offer the user an appropriate 
vocabulary set as s/he encounters the various 
conversation topics, partners, and settings 
throughout the day. As a first step toward this 
goal, the present study sought to determine 
whether vocabulary usage does in fact cluster 
by geographic location and if so, how effective 
are these vocabulary clusters for predicting 
future use in similar contexts? This initial 
exploration, considers the vocabulary usage 
patterns of one able-bodied user in various 
geographically distinct locations. The 
geographic locations are sensed using a global 
positioning system (GPS) and the vocabulary 
usage patterns are mined to extract items that 
the user may be likely to use in future 
interactions in those settings. 
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Figure 1. A screenshot of the iconCHAT interface. © 2007, Rupal Patel. Used with permission. 
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principles discussed, however, are applicable to 
iconic AAC systems as a whole. 


In this paper we focus on the use of geographic 
context to accelerate access to situational 
vocabulary and demonstrate how this may be 
implemented in an icon-based electronic 
communication aid. Given that adaptive 
interfaces which change too often or in 
unpredictable ways can pose additional 
challenges, we chose to display the results of 
the prediction algorithm on our prototype 
communication system called iconCHAT. This 
system was chosen for illustration purposes 
since it provided a test-bed for discussions 
about the competing trade-offs between 
vocabulary size and organization and novel 
visual navigation schemes. For example, given 
that fringe vocabulary makes up only a fraction 
(approximately 1 5%) of the vocabulary used, its 
was essential that users always had access to 
core vocabulary items and that only the 
appropriate fringe vocabulary would be made 
accessible based on context cues. The general 


The iconCHAT system is aimed at enhancing 
communication ease and efficiency by offering 
a semantically organized method of message 
construction and a context-driven method of 
vocabulary organization and prediction (see 
Figure 1). Vocabulary is organized in three 
panels: semantic frames, lexical categories, and 
lexical items. Semantic frames consist of action 
words or verbs that provide the user with the 
scaffolding to construct a message [based on 
case grammars in Fillmore (1968); see also 
scripts in Schank (1973)]. The frame slots 
(displayed in the semantic schema), can be 
filled with lexical items that are classified in 
their respective lexical categories. Once a 
message is constmcted, the user can generate 
spoken output using the text-to-speech 
synthesizer through the control panel. A set of 
message parameters allow the user to directly 
control the contents and expression of his/her 
message. Context parameters inform the user 
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of the system’s sensed location, time, and 
conversational partner [a detailed account of 
iconCHAT is provided in Patel, Pilato, and Roy 
(2004)]. 

We propose using location context to filter and 
organize vocabulary based on previous usage 
patterns. Location has been identified as one of 
the most definitive elements of context in 
ubiquitous computing (Beigl, Zimmer, & 
Decker, 2002). It is possible to determine the 
location of a person with significant accuracy 
using the GPS in outdoor environments. With 
the emergence of the E911 directive (Federal 
Communications Commission, 2005), 
Assistive-GPS (AGPS) technology will allow 
location detection in indoor environments in 
the near future. Endowing an AAC system with 
location-dependant, user-specific vocabulary 
may enhance access to situational vocabulary 
and thereby support timely and appropriate 
message constructions. 

Method 

Data Collection 

An able-bodied, adult male native English 
speaker collected a corpus of spoken data over 
the course of 5 weeks in various geographical 
locations. Approximately 2.7 hours of audio 
was recorded on a daily basis for a total of 20.4 
hours of silence-removed speech collected over 
the 5 week period. It is important to note that 
we began with an able-bodied individual rather 
than an AAC user in order to collect a 
sufficiently large corpus of data necessary for 
building location-dependant vocabulary 
models. Future plans include gathering 
location-sensitive expressive output from a 
group of AAC users. 

We captured the location information using the 
Garmin GPS receiver (Garmin model 35 LVS 
wearable) and the Geostats GeoLogger 
(Geostats, Inc. Model DL-04, Version 3.4). The 
data logger was configured to record latitude. 


longitude, date and time at five second 
intervals. Since currently available GPS does 
not work in indoor spaces, the subject was 
advised to place the receiver near a window or 
any such opening with a clear view of the sky. 

Spoken utterances were captured on the HP 
iPAQ h2215 personal digital assistant (PDA) 
using voice recording software (Resco, sro., 
2007). The on-board microphone of the PDA 
was used with automatic gain control (AGC) 
disabled to minimize ambient noise. The 
software was configured to record date and 
time stamped voice recordings at an encoding 
rate of 16 bits sampled at 22050 Hz. PDA time 
and GPS time were synchronized using the 
U.S. Naval Observatory Master Clock. 

The participant was instmcted to record 
conversational speech in his everyday routines. 
The participant identified certain locations that 
he felt comfortable recording in and was asked 
to frequent these locations for several minutes 
over the course of the week. For reasons of 
privacy and practicality, not all conversational 
contexts were sampled. Location clustering 
revealed eight locations types within his spoken 
corpus: the campus bookstore, a research lab, 
various classrooms, clothing stores, electronic 
stores, grocery stores, the user’s kitchen, and 
his lab meetings. 

Subsequent to data collection, each audio file 
was manually transcribed and coded by 
location. While we attempted to transcribe the 
audio files using automatic speech recognition 
(Nuance Communications, Inc., 2007), we 
found poor accuracy rates due to the level of 
ambient noise in the various natural 
environments. For each of the eight locations, a 
random sample of 20% of the data was 
transcribed by two listeners in order to 
establish inter-rater reliability. Across all 
locations, the average inter-rater reliability was 
98.4%. For words that were not in agreement, 
an additional pass of the transcription was 
completed to resolve any discrepancies. 
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Table 1 

Number of Words and Sentences Per Location 


Location 

N of Words 

N of Sentences 

Bookstore 

7161 

656 

Research Lab 

5666 

474 

Classrooms 

2962 

409 

Clothing Store 

12256 

1122 

Electronics Store 

8205 

687 

Grocery Stores 

9898 

1375 

Kitchen 

8885 

1085 

Lab Meetings 

5761 

493 

Total 

60794 

6301 


Transcribed text and location data were 
integrated into a synchronized representation 
for the data mining and clustering phase. Table 
1 provides a summary of the collected corpus 
in terms of the number of words and sentences 
per geographic location. 

Generating Location-Dependant Frequent Vocabulary 
Patterns 

An association rule-based data mining 
algorithm was used to implement location- 
dependant vocabulary prediction. A pattern is a 
set of items, or sequence of items that reoccur 
within a database. The probability that a 
transaction contains a pattern is referred to as 
its support. A pattern is said to be frequent if 
its support is greater than a predetermined 
minimum threshold and is said to be maximal if 
it has no other frequent super-pattern (Han & 
Kamber, 2000). 


sentential unit (i.e., a phrase, a sentence, or a 
fragment) as a transaction in order to generate 
frequent patterns. 

Prior to further processing we implemented 
stop-word removal to eliminate core 
vocabulary items. Stop-words include words 
such as the, is, and, or, if, to, I, you, this, that, 
etc. which are thought to carry little semantic 
content. In addition to using stop-word lists 
widely used in natural language processing, 
such as the DVL/Verity Stop Word List 
(Defense Virtual Information Architecture, 
2000) and the University of Glasgow Stop 
Word List (van Rijsbergen, 1979) we generated 
a speaker-dependant stop-word list containing 
colloquialisms and words such as like, you 
know, and wow that occurred often in the 
present corpus. These words and phrases were 
used across locations and carried little, if any, 
semantic content. 


We used Borgelt’s adaptation (Borgelt, 2003) of 
the Equivalence Class Transformation (Eclat) 
algorithm (Zaki, Parthasarathy, & Li, 1997), a 
parallel frequent pattern mining algorithm. The 
algorithm reduces the number of passes over 
the database and is highly efficient in 
generating maximal frequent patterns. In our 
implementation, the algorithm considers a 


Following stop-word removal, we ran the 
algorithm using different invocation parameters 
and minimum support thresholds. These 
parameters included the frequent pattern size 
(i.e., one word, two word, etc.) and an option 
for generating only maximal frequent patterns. 
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Classifying Frequent Patterns into Discrete Levels of 
Predictiveness 

Each lexical category in iconCHAT contains a 
one -layer deep lexical item set. For example, 
‘jeans’ is contained in the category ‘clothes’ and 
similarly ‘pizza’ in the category ‘food’. We have 
implemented a three-level differential shading 
scheme to help users scan and select the most 
appropriate vocabulary given their geographic 
context and current stage in message 
construction. 

Decisions on differential shading of lexical 
items and semantic frames are based on 
frequency statistics observed within each 
location. The algorithm generates a rank 
ordered list of size-one (i.e., one word) frequent 
patterns. The support value of each frequent 
item is used to sort the list into one of three 
discrete levels: highly probable, probable, and 
neutral, conditioned on the user’s current 
location and his/her current state in message 
construction. These discrete levels are visually 
represented using an arbitrarily defined 

grayscale shading scheme where the lightest 
items are those from the highly probable 
category to facilitate easy access. For example, 
‘coke,’ ‘fries,’ ‘drink,’ ‘hamburger,’ and ‘small’ 
may be the top five (size-one) frequent times in 
a restaurant. Each item would have an 
associated support value that is used to 

determine whether the item should be 

categorized as highly probable, probable, or 
neutral. 

Generating Quick Access Vocabulary Based on 
Frequent Patterns and Semantic Kelatedness 

To further improve vocabulary access we 
employed frequent patterns and semantic 

relatedness measures to populate a quick access 
vocabulary panel. Semantic relatedness is a 
quantitative measure of the degree to which 
two words are related. We used this measure to 
select closely related lexical items from the 
frequent pattern inventory. Several measures 


have been proposed to assess the semantic 
relatedness using WordNet (Fellbaum, 1998). 
WordNet is an online database in which lexical 
items are represented as a synonym sets. 
Lexical items in the database are linked to one 
another by approximately 30 semantic relations. 
Budanitsky and Flirst (2001) compared five 
methods of implementing relatedness using 
WordNet and found that the Jiang and Conrath 
(JCN; 1997) measure was most suited for 
natural language processing tasks such as ours. 

1. We implemented a multi-step algorithm 
to populate the 3x8 icon quick access 
panel: 

2. Select all size-one frequent patterns that 
have support values higher than the 
threshold. 

3. For maximal frequent patterns of size 
two or greater, select all remaining 
items. 

4. Given that the quick access panel can 
only accommodate a maximum of 24 
icons, calculate the number of vacant 
icon slots after steps 1 and 2. 

5. Calculate semantic closeness using the 
JCN measure (Pedersen, Patwardhan, 
& Michelizzi, 2004) for all pairs of size- 
two patterns that contain at least one 
element from step 1 . 

For each item in step 1, calculate the number of 
vacant slots that it can contribute to the panel 
based on its support. Fill these slots with 
item(s) that are most semantically related to it. 

This algorithm yielded a list of lexical items that 
were used to populate the quick access panel 
for each unique geographic location. 

Results and Discussion 

In this section, we present the results of the 
algorithms described above and elaborate on 
how the findings can be leveraged to improve 
vocabulary access in an iconic AAC system. 
Interface adaptations are discussed in terms of 
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the iconCHAT system to provide concrete 
examples of possible implementations. The 
general principles discussed, however, are 
applicable to AAC systems as a whole. We 
discuss issues of visual organization and 
navigation speed that influence message 
construction and their implications on human 
versus machine generated vocabulary selection. 


kitchen, onion has a support value of only 
39.5). This indicates that in locations such as 
the bookstore, the word book is likely to be part 
of a greater number of sentences, while onion 
may appear in fewer sentences produced in the 
kitchen. Recall that high support values 
influence the contribution weight of the item in 
the quick access panel. 


Frequent Items in Each Eocation 

A partial list of the 20 most frequent size-one 
patterns in the spoken corpus in each of the 
eight locations is presented in Table 2. Note 
that for some locations, the most frequent 
pattern has a relatively high support value (i.e., 
in bookstore, book has a support value of 69) 
while in other locations the most frequent 
pattern has a lower support value (i.e., in the 


Additionally the range of support values in the 
frequent patterns also differed by location (e.g., 
for Table 1, the bookstore support value range 
= 55.2; kitchen support value range =18.4). 
This range influenced the cutoff values for 
classifying items into the three discrete 
likelihood levels for differential shading. 
Frequent patterns that occur across multiple 
locations were typically core vocabulary items 
such as predicates (want, look, etc.), modifiers 


Table 2 

Size-One Frequent Patterns in Each Location and Their Support Measures 


Bookstore 

Research Lab 

Classrooms 

Clothing Stores 

Electronics Stores 

Grocery Stores 

Kitchen 

Lab Meetings 

book-69.0 

thing-47.7 

time-50.0 

Look-56.9 

look-55.7 

need-63.0 

onion-39.5 

time-44. 1 

look-41.4 

people-45.5 

thing-42.9 

Nice-52.0 

nice-28.6 

look-40.7 

cut-36.8 

way-38.2 

class-39.7 

rime-43.2 

motor-35.7 

Shirt-46.1 

thing-28.6 

make-37.0 

use-36.8 

try-38.2 

stuff-34.5 

work-34.1 

way-35.7 

color-36.3 

stuff-24.3 

want-37.0 

carrot-34.2 

little bit-35.3 

need-32.8 

say-31.8 

people-35.7 

jacket-31.4 

probably-24.3 

thing-33.3 

soup-31.6 

different-32.4 

thing-31.0 

class-31.8 

little bit-28.6 

stripe-30.4 

little-24.3 

Cheese-31.5 

water-31. 6 

say-32.4 

time-31.0 

sort-29.5 

stuff-28.6 

Wear-29.4 

small-22.9 

ones-29.6 

chop-28.9 

class-29.4 

read-25.9 

speech-27.3 

better-21.4 

Stuff-26.5 

work-20.0 

stuff-27.8 

little-28.9 

record-26.5 

use -24.1 

little bit-27.3 

play-21.4 

Blue-26.5 

sort-17.1 

use-25.9 

need-28.9 

minute-29.4 

buy-20.7 

use-25.0 

oral-21.4 

pants-25.5 

different-17.1 

cream-24.1 

parsley-26.3 

able-26.5 

different-19.0 

person-22.7 

facial-21 .4 

Ones-24.5 

speaker-17.1 

chicken-22.2 

add-26.3 

easy-23.5 

probably-19.0 

different-22.7 

sure-21.4 

sale-24.5 

want-15.7 

probably-20.4 

little bit-23 .7 

recording-23.5 

speech-19.0 

disorder-20.5 

disorder-21.4 

different-24.5 

sure-14.3 

better-18.5 

garlic-23.7 

want-23.5 

way-17.2 

language-20.5 

talking-21.4 

white-23.5 

little bit- 1 4.3 

sure- 16.7 

pepper-23.7 

data-23.5 

sort- 17.2 

want-20.5 

day-21.4 

sort-22.5 

player-14.3 

ginger-16.7 

boil-23.7 

thing-23.5 

sure- 15.5 

talk-20.5 

couple-21.4 

Light-22.5 

big-12.9 

little bit- 14.8 

thing-21.1 

file-23.5 

actually- 15.5 

semester-20.5 

want-21.4 

probably-21.6 

digital-12.9 

plantains-14.8 

fresh-21.1 

sort-20.6 

a_lot-15.5 

lab-18.2 

prepare-14.3 

black-21.6 

watch-12.9 

list-14.8 

way-21. 1 

make-20.6 

funny- 13.8 

start-18.2 

words- 14.3 

Thing-18.6 

use-12.9 

feel-13.0 

vegetable-21.1 

work-20.6 

textbook-13.8 

difficult- 18.2 

read-14.3 

Little bit-1 8.6 

regular-12.9 

eat-14.8 

celery-2 1.1 

maybe-20.6 
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Table 3 

Impact of User-Dependant Stop-Word Pruning on Frequent Patterns in the Clothing Store 
Context 


Pruned Frequent Patterns 

Generic Stop-Word User-Dependent Stop-Word 


like - 83.7% 

look - 56.9% 

kind - 69.6% 

nice - 52.0% 

know - 47.4% 

shirt - 46.1% 

look - 45.9% 

color - 36.3% 

yeah — 45.9% 

jacket - 31.4% 

nice - 43.7% 

stripe - 30.4% 

oh - 38.5% 

wear - 29.4% 

just - 37.0% 

stuff -26.5% 

shirt - 34.8% 

blue - 26.5% 

color - 29.6% 

pants - 25.5% 


(a little bit, a lot), or generic nouns (thing, 
stuff). Perhaps these items should be made 
accessible in a different part of the interface 
such as in a context-independent panel of scroll 
buttons. 

Factors that Influenced Algorithm Performance 

Given that stop-words occur often in spoken 
dialogue, they may alias as frequent patterns. As 
a result, other more content-laden vocabulary 
items may be pushed lower in the frequent 
pattern ranking or lost completely. In addition 
to using conventional stop-word lists, we found 
that a speaker-dependent list greatly enhanced 
performance (see Table 3 for a truncated list of 
size-one frequent items in the clothing store 
generated with and without user-dependant 
stop-words). An effective stop-word removal 
phase is especially important for iconCHAT 
since the natural language generation module 
attempts to minimize keystrokes by allowing 
the user to only select content words, leaving 
out function words such as of, to, and the during 
message construction. 


Transaction size also had a significant impact 
on algorithm performance. Conversational 
speech typically contains less formal structure 
than written text. Incomplete grammatical 
constmctions, fragments, lengthy trains of 
thought and backchannel exchanges often 
characterize typical spoken conversations. In 
contrast, communicative exchanges using 
iconCHAT fall somewhere between 
conversational dialogue and written discourse. 
Keeping these factors in mind, we 
experimented with varying lengths of sequential 
units to arrive at an ideal transaction size. 

Leveraging Frequent Patterns to Provide Visual 
Selection Cues 

For each location, size-one frequent patterns 
were classified into three discrete predictive 
levels within each lexical category. Predictive 
shading of category items enables the user to 
rapidly scan and sift through the most likely 
lexical items within each category. Figure 2 
illustrates this feature in the clothes and colors 
categories for the clothing store context. 
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Figure 2. Location-dependant predictive shading for clothes and colors categories. © 2007, Rupal 
Patel. Used with permission. 



In the clothes category, the algorithm ranked 
jacket and pants to be among the most probable 
items and were thus represented at the 
brightest, most accessible level. Conversely, 
words such as belt and dress appear darkest and 
least accessible since they were judged to be 
neutral items for this user. Note that within 
each category, lexical items are alphabetically 
arranged as an attempt to assist in symbol 
search and promote early literacy skills. 

Location-Dependant Quick Access Vocabulary 

The quick access panel is aimed at further 
improving message construction ease. In other 
words, it helps reduce the number of clicks 
required to create a message by providing the 
most likely lexical items for a given context 
within a single panel (see Figure 3). Closer 
examination of the lexical items reveals their 
diversity with respect to their category and part 
of speech roles. In addition, an item such as 


‘dress’, which was ranked 47 in the size-one 
frequent patterns, is promoted to the quick 
access panel containing only 22 members 
because support values and semantic closeness 
measures were considered together in this stage 
of analysis. Just as with differential shading, we 
anticipate that after repeated exposure, the user 
may gain insights into his/her own vocabulary 
usage patterns. The contents of the quick 
access panel may gradually change over time. 
For example, in the clothing store context, 
frequent items may change over the seasons. 
These changes occur over extended periods of 
time (e.g. weeks, months) such that the user is 
not confronted with a continual changing 
interface. Also note that the user still has access 
to all vocabulary items through the lexical 
category panel if the contents of the quick 
access panel do not meet his/her needs. 
Perhaps these seamless changes in the 
vocabulary content will promote receptive 
language and categorization skills. 
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Figure 3. Quick access panel for the clothing store location. © 2007, Rupal Patel. Used with 
permission. 



Correlating Human and Algorithmic Rankings of 
Predictive Vocabulary 


and algorithm rankings provide evidence that 
actual versus perceived frequency statistics 
often differ. 


The Spearman ranked correlation measure was 
used to compare algorithm-generated 
vocabulary usage patterns with the user’s 
perception of his patterns. Rankings were 
requested for the 10 most frequent size-one 
items in each location (see Table 4). Rankings 
for only nouns and modifiers were requested 
given our interest in assessing the effectiveness 
of predictive shading and quick access. 

Inspection of the correlations across locations 
indicated that in general, locations with a large 
support value range in frequent patterns (i.e., 
bookstore) had higher correlations than those 
with a reduced support range (i.e., kitchen). 
From the user’s perspective, it seems intuitive 
that a distribution of support values over a 
large range would allow more precise ordering. 
Conversely, a smaller range in support value 
distribution would make it difficult for the user 
to discern the importance of one item over 
another resulting in low or negative 
correlations. These discrepancies between user 


Trained clinicians, such as speech language 
pathologists, often make vocabulary choice 
decisions when programming AAC devices 
based on their own experiences and intuitions 
about a user’s needs. Using an algorithmic 
approach to make these decisions based on 
actual frequency statistics along with clinical 
insights may improve communication 
efficiency and effectiveness. The algorithm may 
identify subtle, idiosyncratic patterns that are 
specific to that user thereby improving device 
customization. Since many users prefer a 
certain degree of control over their system 
rather than delegating decision making to the 
system (or to a clinician), a hybrid approach 
may balance these tradeoffs. 

Outcomes and Benefits 

This paper described an initial effort 
aimed at leveraging situational context for 
vocabulary prediction. The methodology 
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Table 4 

Correlation Between User and Algorithm-Generated Rankings Size-One Frequent Patterns 


Locations 

Spearman Rank Correlation Measure 

Bookstore 

0.733 

Research Lab 

0.479 

Classrooms 

0.491 

Clothing Stores 

0.418 

Electronics Stores 

0.346 

Grocery Stores 

0.079 

Kitchen 

-0.369 

Lab Meetings 

-0.212 


described mined vocabulary usage patterns of 
one able-bodied male user in various 
geographic contexts. The resulting context- 
dependant vocabulary patterns were 
implemented on the iconCHAT prototype 
system to illustrate how vocabulary access may 
be enhanced. While we began with an able- 
bodied user to ensure sufficient data on which 
to implement and assess the data mining 
algorithm, this open-set corpus had its 
drawbacks. The conversational topics, types of 
message constructions, and vocabulary choices 
were diverse and complex. Given that sentence 
constructions, conversational topics and 
frequent locations will vary on an individual 
basis additional algorithm tuning will be 
required in future implementations that operate 
on usage patterns of a typical AAC user. For 
some AAC users, the vocabulary set on their 
device may restrict the type and number of 
unique messages that can be created. These 
constraints may in fact improve algorithm 
performance. 


was poor. A hybrid approach would entail the 
algorithm suggesting a vocabulary set (based on 
past usage patterns) from which the user or 
trained professional could choose the relevant 
items. Selecting the appropriate vocabulary is 
key to achieving improved communication 
success. 

Another area that can significantly impact 
vocabulary access and thereby communication 
effectiveness is the user’s understanding of 
his/her own usage patterns. Thus, if a user 
knew that frequent lexical items which are 
represented at the most accessible grayscale 
level were also likely to be in the quick access 
panel, s/he may be able to efficiently navigate 
to the desired item. In other words, knowing 
where to find a particular vocabulary item may 
accelerate access by reducing the number of 
keystrokes for message construction. Usability 
studies that assess the impact of situational 
vocabulary prediction on keystrokes savings, 
cognitive load and message construction 
accuracy and complexity are warranted. 


Correlations between the user and algorithm- 
generated frequent pattern rankings suggest 
that decisions about vocabulary sets may 
benefit from a hybrid approach. In some 
locations the user’s perception of frequent 
patterns were highly correlated with the actual 
statistics while in other locations the correlation 


Limitations and Future Directions 

We plan to extend this work by collecting a 
more extensive corpus of message 
constmctions produced by a group of able- 
bodied users as well as a group of AAC users 
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within varied geographic contexts. Particular 
attention will be given to collecting sample 
utterances in a variety of conversational settings 
with multiple communication partners. This 
corpus will then be used to generate context- 
dependant vocabulary patterns for each user. 
Subsequently, we will evaluate the usability of 
discrete predictive levels of shading and the 
quick access panel. 

We plan to broaden the methodology by 
building user-specific predictive vocabulary 
networks. Such networks would provide insight 
into organizational and navigational features 
that are beneficial across users. These insights 
can then be incorporated into future iterations 
of the interface design. 

While this paper focused on the use of 
geographic context for vocabulary prediction, 
we are currently working toward harnessing 
other contextual cues such as time of day and 
conversational partner and topic. A significant 
challenge will be to integrate and visually 
represent vocabulary prediction using these 
varied contextual cues. Additionally, it will be 
important to monitor the use and effectiveness 
of situational vocabulary prediction. 
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